A Practitioner’s Guide to MDP Model Checking Algorithms

Abstract Model checking undiscounted reachability and expected-reward properties on Markov decision processes (MDPs) is key for the verification of systems that act under uncertainty. Popular algorithms are policy iteration variants value iteration; in tool competitions, most participants rely latter. These generally need worst-case exponential time. However, problem can equally be formulated as a linear program, solvable polynomial In this paper, we give detailed overview today’s state-of-the-art MDP model with focus performance correctness. We highlight their fundamental differences, describe various optimizations implementation variants. experimentally compare floating-point exact-arithmetic implementations all three benchmark sets using two probabilistic checkers. Our results show (optimistic) sensible default, but other preferable specific settings. This paper thereby provides guide practitioners—tool builders users alike.